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Abstract 


With increased availability of information in modern societies, individuals are often faced with 
complex decisions regarding how to integrate and judge the veracity of available information. 
Generally, these issues have been approached using computational techniques to detect and 
reduce the spread of information across social media. However, few researchers have examined 
theoretically-grounded characteristics of misinformation. The purpose of this chapter is to 
examine this phenomenon through the lens of discourse processing theories, which emphasize 
interactions among features of the discourse, the reader, and the context. We describe a proof of 
concept on how dynamical systems modeling combined with computational linguistics has 
strong potential to reveal underlying characteristics of the spread of misinformation. 
Additionally, we discuss potential directions for future research, 

as well as implications for interventions to help students accurately process information in the 
modern digital age. We call for research using a combination of computational linguistics, 
telemetry, and dynamical systems analytics in order to better understand the temporal 


organization of text and the spread of misinformation. 
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online news and social media (e.g., emails, blogs, tweets, 
the 24-hour news cycle) in our modern society has dramatically increased 
individuals’ access to information. One consequence of this cultural shift 
is that sources of information are widely varied in their quality and ve- 
racity; thus, individuals are often faced with complex decisions regarding 
how to evaluate and integrate these sources. Unfortunately, this task does 
not come easily to all. An estimated 17% of adults in the United States 
are illiterate (nces.ed.gov/surveys/piaac/results/summary.aspx), and the 
majority of students graduating from high school are ill-prepared to com- 
prehend, integrate, and evaluate information from complex texts (NAEP, 
9015; OECD, 2013). Thus, an important question facing both educators 
and researchers is how to train individuals to understand, learn from, and 
critically evaluate text content in the Internet-age (Braasch, Wiley, Geae- 
ser, & Brodwinska, 2012). 

One response to this issue has been an increase in research examining 
how individuals process information that is presented to them across mul- 
tiple texts or documents (Braasch, Braten, & McCrudden, 2018; Goldman, 
2004; Goldman et al., 2016). This research is motivated by the widespread 
availability of texts in modern, technology-driven societies, as well as the 
a of daily tasks (e.g., dietary and medical decisions, evaluation of 
mage a: oS rely on processing and learning from multiple a 
outa ; , Single text comprehension, the process of comprehending 
multiple documents requires individuals to make connections betwee? 
information contained withi : i-document 
coimiprehension lies ¢ within texts. The difference for multi Oo a 
various documents ch a additional task of generating ny eer. € 
and skills to ivi aa that individuals often lack sufficient know oe 
they are faced with ie single texts, their problems are compounded W 
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beenadramatic incre and demonstrate the urgency of this issue, pees di- 
Viduals to Sway aikteae inthe use of social media by political groups and in 
pinion through the dissemination of misinformatio™ 


The prevalence of 


ae 


Misinformation Through Discourse Dynamics = 161 


rumors, and propaganda. For instance, botnets (i.e., networks of computers 
that have been programmed to automatically perform specific tachi 
been extensively relied on to spread misinformation across social medi ties 
(Agarwal, Al-khateeb, Galeano, & Goolsby, 2017; Woolle 90 ae 

. ; ey, 2016). A recent 
report estimated that 8.5% of Twitter and 7% of Facebook accounts (over 50 
million accounts) are fake (Lokot & Diakopoulos, 2016). Further, many of 
these fake accounts are run by highly sophisticated bots that try to a 
man behaviors, which makes their detection quite challenging. 

Researchers have generally addressed this issue through the develop- 
ment of computational techniques to detect and reduce the spread of misin- 
formation (e.g., Bajaj, Kavidayal, Srivastava, Akhtar, & Kumaraguru, 2016; 
Qazvinian, Rosengren, Radey, & Mei, 2011). For example, a common ap- 
proach involves the identification of keywords within texts (e.g., headlines, 
texts, tweets) to develop algorithms that can detect misinformation. Al- 
though several of these approaches have been successful, a primary ob- 
stacle to their implementation is that the production of fake news content 
is consistently evolving in response to the development of these detection 
tools. Thus, individuals cannot simply be taught to identify misinformation 
based on specific features (e.g., linguistic properties) of texts, headlines, or 
social media accounts. Instead, it is important that researchers identify the 
specific cognitive processes that underlie the evaluation and uptake of mate- 
rial in fake news sources so that individuals can adapt with these evolving 
production strategies. This points to a significant gap in the field that re- 
mains to be addressed. Namely, few researchers have examined the factors 
involved in the spread and uptake of misinformation, such as the linguistic 
content of misinformation and the dynamics of the cognitive processes and 
strategies that are employed to detect false information in texts. 

The purpose of this chapter is to examine these processes through the 
lens of discourse processing and dynamical systems theories, which togeth- 
€r emphasize complex interactions among properties of discourse, read- 
€rs, and contexts. We first provide an overview of these theories along with 
a review of state-of-the-art methods for automated text analysis. Next, we 
Propose an integration and extension of these fields that accounts for how 
complex phenomena arise and unfold over time. In particular, we draw 
on theory and methodologies from dynamical systems theory to develop 
at oa of concept on how the spread and processing at ae i 
a Ae modeled. Finally, we discuss potential directions for a 
donk » aS well as implications for educational anteryentons to help stu- 

accurately process information in the modern digital age. 
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in interconnected and related to prior knowledge (Kintsch, 1988, 1998; 
qwaan & Radvansky, 1998). : . 

Readers generate inferences to establish how text constituents (e.g., pro 
ositions) are related to one another (Graesser et al., 1994; Kintsch, 1988), 
For example, bridging inferences establish how two elements of a text are 
semantically connected, whereas knowledge-based inferences involve the 
use of prior knowledge to elaborate on text content (McNamara & Maglia- 
no, 2009a, 2009b). In turn, the relations established by the reader can be 
at relatively shallow levels (e.g., overlap between words) or deeper seman- 
tic levels (e.g., causal, spatial-temporal; Wolfe, Magliano, & Larsen, 2005), 
These processes support the construction of a coherent mental model (Mc 
Namara & Magliano, 2009b). 

Prior domain knowledge can help to facilitate text comprehension, and 
particularly the process of generating inferences. Readers who have more 
knowledge about the topic of a text are able to process the information 
more quickly, remember more of the information, understand the infor 
mation at a deeper level, and more effectively ignore irrelevant informa 
tion (e.g., Alexander, Kulikowich, & Schulzse, 1994; Bransford & Johnson, 
1972; Chiesi, Spilich, & Voss, 1979; Haenggi & Perfetti, 1994; McNamara & 
Kintsch, 1996; McNamara & McDaniel, 2004). When readers have limited 
prior knowledge of the topic, the construction of a coherent mental model 
can be hindered. Moreover, when the text is challenging (e.g., difficult vo- 
cabulary, complex syntax, low cohesion), the negative effects of knowledge 
pith be exacerbated (McNamara, O'Reilly, Best, & Ozuru, 2006). 
Rens there are ways of altering a text itself to make it more ei 
in the « (eg, increasing cohesion, providing greater links to the wor i 

teal world students are faced with an abundance of challenging ®* 
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Research se that skilled readers more effec 
«on texts primarily because they are more }; 5 

“ie “i These strategies, such as comprehension comprehension ane 
and generating predictive, bridging, and elaborative inereron me 
dents to connect information ACTOSS a text (Magliano, Trabasso, & Gr ack 
1999; McNamara & McDaniel, 2004, O'Reilly & McNamara 2007a 2007b) 
prompting students to selfexplain a text, that is, instructing she és ex on 
the text to themselves as they read, has been shown to increase the is of 
comprehension strategies and, consequently, improve reading comprehen- 
sion (Chi, de Leeuw, Chiu, & LaVancher, 1994). Further evidence sil 
that students can be taught to use comprehension Strategies (Brown, 1982; 
Palincsar & Brown, 1984). Similarly, the quality of students’ self-explanations 
improves with comprehension strategy instruction and practice. 


tively tackle low cohe- 


Comprehension of Multiple Documents 

With the Internet now serving as a primary source of learning and in- 
formation for individuals, there has been a dramatic increase in research 
on the factors that influence the comprehension of multiple documents 
(Braasch et al., 2018). The majority of this research has focused on how 
individuals critically evaluate the sources of information that are presented 
to them across texts. This is a particularly important issue, as a wealth of 
false information is now available across a wide variety of sources on the 
Internet. Source evaluation relies on a complex set of processes and skills 
that require the reader to process features of the sources (e.g., author, pub- 
lication venue) to make informed decisions about the quality and accuracy 
of information in a given text or set of texts (Braten, Stadtler, & Salmerén, 
2017). Research has shown that actively attending to source information 
is crucial to the successful integration of information across documents 
(Brand-Gruwel, Wopereis, & Walraven, 2009; Britt, Rouet, & Braasch, 2013; 
Lawless, Goldman, Gomez, Manning, & Braasch, 2012; Perfetti, Rouet, & 
Britt, 1999; Rouet, 2006; Rouet & Britt, 2011). However, in the absence of 
domain expertise, individuals do not readily engage in these source evalua- 
4on processes (Wineburg, 1991), and the effects of source evaluation train- 
ing have been mixed (e.g., Wiley et al., 2009). 


Automated Analyses of Text and Discourse 


One way that researchers have attempted to examine the text compre- 
hension processes is to identify and model the linguistic properties of texts 
that influence comprehension. For instance, words that are more frequent 
in the English language tend to be more familiar to readers and a nage 
fore, processed more quickly and deeply by readers (Beck, - = 

ucan, 2002; Haberlandt & Graesser, 1985; Perfetu, 200%), Batons 
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- ouistic properties provide researchers with the mea ayeloped the Interactive Strategy Training for Act : ; 
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language an of language as they relate to particular tasks. NLP tools algorithms also provide feedback to help students develo 
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: +t including coherence, syntactic complexity, lex; 

ing and producing oe slarity, One common approach is i a 
oe aerindividval words, or 7grams (i.e., groups of words where n, re- 
os wee number of grams included in the group; Jarvis & Crossley, 2012), 
ao word and n-gram calculations allow researchers to examine the ex- 
plicit content of a text. A different approach with a stronger potential to 
generalize involves the calculation of the linguistic features of the words 
and sentences in a text (McNamara, Allen, Crossley, Dascalu, & Perret, 
9017). We have developed several NLP tools such as Coh-Metrix, which can 
be used to extract linguistic features of text across multiple levels. 
Using NLP, we can derive features of students’ constructed responses 
that are indicative of the cognitive processes that occur during learning 
(Graesser & McNamara, 2011). Over the past 2 decades, there have been 
substantial advances in the application of NLP techniques to'support anal- 
yses of constructed responses (Landauer, McNamara, Dennis, & Kintsch, 
2007; Shermis, Burstein, Higgins, & Zechner, 2010). These advances have 
been in the context of computer-based assessments of think-alouds and self 
explanations (Gilliam, Magliano, Millis, Levenstein, & Boonthum, 2007; 
Magliano et al., 2011), grading of essays (Attali & Burstein, 2006; Burst- 
ein, 2003; Burstein, Marcu, & Knight, 2003; Landauer, Laham, & Foltz, 
2003; Shermis et al., 2010), grading of short answer questions (Leacock & 
Chodorow, 2003), and intelligent tutoring systems and trainers that require 
students to produce responses during interactive conversations (McNama- 
ra, Levinstein, & Boonthum, 2004). 
: bec irae a number of automated systems and tutors that incor 

. oak e LP tools and algorithms to assess constructed responses, 2m 
solving pe about student comprehension, learning, and problem 
Strategy Ass example, Magliano and colleagues have developed Reading 
‘ssment Tool (RSAT; Magliano et al., 2011). RSAT involves hav- 


: ; these strategi 
and produce higher quality self-explanations, : teen 


Indeed, computational linguistic techniques can serve as a powerful 
methodology for modeling individual differences and specific processes in 
which students engage (Rus, McCarthy, Graesser, & McNamara, 2009). To 
better understand the relations between linguistics and cognitive processes, 
we can first consider the notion that there are multiple linguistic features 
of texts that students comprehend and produce. These features impact the 
quality and readability of a given text as a function of the prior knowledge 
and skills of a given learner (Graesser & McNamara, 201 1); 
There are a multitude of linguistic features and dimensions to character- 
ize text and discourse. Nonetheless, one heuristic is the consideration of 
two broad categories: surface-level and discourse-level features. Surface-level 
features relate to the characteristics of the words and sentences. Variations 
in these features can alter the style of the text, and influence its readability 
and perceived sophistication. For example, the frequency that a word oc- 
curs in English is linked to the familiarity of that word, as well as how quickly 
it is processed and how strongly it is linked to rich bodies of knowledge 
(Beck et al., 2002; Haberlandt & Graesser, 1985; Perfetti, 2007). Thus, texts 
that contain more frequent words will typically be easier than texts with less 
frequent words. 
Discourse-level features go beyond the individual words and sentences, 
and reflect aspects of the mental model such as the degree of narrativity 
in the text and the strength of the relations (i.e., cohesion) between ideas 
across the text (Graesser & McNamara, 2011). These features influence the 
€ase at which the text can be understood and recalled, even when control- 
ling for variance associated with students’ familiarity of the topic and the 
difficulty of the words (Haberlandt & Graesser, 1985). 
‘ee the differential effects of surface- and discourse-level linguistic 
ures On text processing and comprehension, it follows that these features 


ing students produc x ola Vary across constructed responses as a function of learners’ individ- 
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Ms exhibit amara, 2014), comprehension skills (Allen, Snow, & McNamara, 2015), 


Protocols), convergent eas (correlate with human coding of 


hension proficiene ) Correlate with independent measures of COmPI” o Writing skills (Crossley & McNamara, 2012; Varner, Roscoe, & McNa- 
ated measures of mh and predictive (correlated with experimenter gene ara, 2013). These results suggest that NLP techniques can inform assess- 
Mprehension) validity. McNamara and colleagues hat 


m : 
€nts and help to improve adaptivity in educational systems. 


166 "= 1.K. ALLEN, A. D. LIKENS, and D. S. MCNAMARA 
Dynamical Systems Theory and Discourse Processing 

A caveat to the strength of NLP pertains to a limitation with regard 
detection of change over time. There are theoretical and practical ie the 
tions in that current NLP algorithms identify evidence of comprehe Mita. 
and skills, but do not provide information about how they occur ite 
bination to support comprehension and learning, or how these praca 
dynamically unfold. We call for research that assesses the Utility of se 
putational techniques based on dynamical systems theory to analyze aa 
temporal organization of students’ responses to multiple texts (including 
authentic texts and texts with misinformation), potentially in conjuasena 
with other relevant time-varying data (e.g., ratings of engagement or ‘ 
strokes during typing). i 

Importantly, dynamical systems theory subsumes a large analytical tool. 
box, which provides a novel means with which researchers can characterize 
patterns that characterize students’ behaviors (e.g., language, system choic- 
es) during learning tasks. Traditional inferential statistics often aggregate 
variables across time, potentially discarding important information about 
learning and performance. In contrast, dynamic methodologies consider 
change over time to be a critical component of the analysis and explicitly 
seek to characterize temporal patterns. Thus, rather than treating behavior 
as a Static process, these dynamic analyses have strong potential to more 
precisely account for the complex, changing nature of behavior. Although 
no studies have applied dynamic analyses to multiple text comprehension 
and misinformation detection, these techniques have previously been used 
across a wide variety of domains as a means to understand the complex 
patterns that manifest in individuals’ behaviors over time (e.g., Anderson, 
Bischof, Laidlaw, Risko, & Kingstone, 2013; Dale & Spivey, 2005; Riley, Bala- 
subramaniam, & Turvey, 1999; Shockley, Santana, & Fowler, 2003). 


it is im- 


Dynamical Systems Theory 
to this 


In order to illustrate the importance of dynamical systems theory, 
portant to understand a number of the key concepts that are important : 
line of research. Dynamical systems theory (DST) provides a principled theore 
ical framework for examining complexities associated with comprehens! oe 
of complex information (Dale, Kello, & Schoenemann, 2016). mh 0 
‘ystems are composed of multiple components that interact over time C a 
1995). Importantly, the complex patterns produced by these systems Ke 
be explained by simply reducing them to their component parts. I ne - 
these patterns emerge through a process of selforganization. That 1s, ee 
erlevel patterns emerge, stabilize, change, and dissipate as a natural CO! ts 
si of local interactions among the system’s components and cons Cee 
E aced on the system. Constraints may be random fluctuations ai by 

om the environment or so-called nonspecific control parameters, parame 


: bu 
tive changes in the system’s behavior—refens noncontinuous, qualita- 
DST approach to understanding human behay; 10 as phase transitions, The 
mains involving physiological tetas Pa avior is most well known in do- 
provided insight within the context of othe motor control, but it has also 
activity (Tognoli & Kelso, 2014), diseonre on domains such as brain 
Tylen, 2014), performance variability {Hateson b re Raczasvek-Leonardi, & 
coordination (Gorman, Amazeen, & Cooke, 2010 tergiou, 2015), and team 
A simple illustration can help to make pes d 
commonly referenced system in the motor nies aie — Paps 
emerges with alternating swinging of limbs, such ee Ntnlipas is one that 
(Amazeen, Amazeen, & Turvey, 1998; Haken, Kelso, & ee ae 
& Turvey, 1987). At slow speeds, two patterns dominate oars i ora 
where the angle between the limbs is 0°, and an Alphas afte 
. . pattern where 
the angle between the limbs is 180°. However, faster speeds result in a ph, 
transition such that only one pattern, 0°, remains stable. In this sim ie Me 
ample, the phase relations between the limbs are the higher-order eee 
that emerge from the local interaction of the systems’ components—here, 
the limbs. The singular control parameter is the speed at which the limbs 
are moving. When the speed of the limbs’ movement reaches a critical 
point (i.e., a critical speed), the system exhibits a qualitative change in 
behavior: the elimination of the 180° pattern and the movement towards 
the inphase pattern. This simple system captures several properties of a dy- 
namical system (e.g., control parameters, phase transitions; Gilmore, 1981; 
Kelso, 1995). This example is not provided as a template for the dynam- 
ics expected to emerge from either students’ natural language patterns, as 
these processes generate patterns that are far more complex (e.g., Allen, 
Perret, Likens, & McNamara, 2017; Dale & Spivey, 2005; Likens, Allen, & 
McNamara, 2017). Nonetheless, the example tacitly emphasizes the impor- 
tance of how higher order patterns emerge as a result of how dynamical 
systems change over time, a feature that plays an important role in the iden- 
tification and analysis of patterns in students’ natural language responses. 
Importantly, the processing and integration of information within and 
across texts does not consist of uniform processes; the nature of these pro- 
cesses changes as a function of the features of the text, metacognitive states 
of the reader, and the relevant prior knowledge that can beaut © oc 
: & van den Broek, 2007; 
ate explanations about the text content (Kendeou ail cone 
McNamara & Magliano, 2009a). We argue that a DST approach will con 
. é ve processes that 
tribute to a better understanding of the complex ii a oe 
underlie the evaluation, processing, and production . _ a a carte 
argument follows the assumption that language me interacting compo- 
duction are driven by complex, dynamical systems W! 


nents and complex emergent properues. 
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An underlying ee aapoiidorl (Kelso, 1995). This has been ot 
Bon “ls 4 ee similitude. One objective of DST is to examine the pres. 
ferred to A of dynamic similarity between systems that may appear to be 
ence and sae on static measures (€.g-, different material substrates Within 
different at f engineering). For instance, the Haken-Kelso-Bunz (HKB) 
the peta et al., 1985) was developed to characterize the rhythmic ¢o. 
model i. of index fingers on the opposing hands of a single individual, 
om predictions of the HKB model were informally presented earlier 
in the discussion of hallmark or of a dynamical system, namely, the 
presence and stability of 0° and 180° coordination patterns. Given that the 
same neuromuscular system that moves the fingers also moves the wrists and 
legs, it is not surprising that the HKB model has been generalized to other 
limbs within a single individual (see Amazeen, 2018; Amazeen et al., 1998; for 
reviews). The HKB model has also been extended to phase synchronization 
patterns observed in brain dynamics (Bressler & Kelso, 2001). 

More remarkably, however, the HKB model has been generalized to 
settings where the coordinated elements do not share a common neryous 
system. For example, Schmidt, Carello, and Turvey (1990) demonstrated 
that the stable patterns (0° and 180°) and accompanying phase transitions 
(annihilation of the 180° pattern) predicted by the HKB model also hold 
when two people rhythmically coordinate the swinging of their limbs. The 
number and variety of instances in which this model has been applied is ex- 
tensive and well beyond the scope of this chapter. These examples provide 
only a hint of its generality. We emphasize this point because the HKB mod- 


el’s ability to characterize complex forms of coordination stands as a para-_ 


mount example of the principle of dynamical similitude—systems ranging 

from clusters of neurons to clusters of people exhibit similar dynamics. . 
Despite the remarkable utility of the HKB model, we do not wish to give 
the impression that it is the only model in the DST toolbox. After all, the 
model was designed to characterize rhythmic movement patterns, which 
are only a small subset of the many behaviors typically found in human per 
formance. Another example of dynamical similitude comes from the appli- 
cation of fractal analysis, a form of dynamic modeling that has been used to 
Phas complex systems emerging from the interaction of multiply-nested 
ak Ste & Vereijken, 2010). The systems that have been studied 
Sina neigtta ene Over several orders of magnitude, ranging from a, 
ation (Picol, fiends ig a & Bullmore, 2017) to geomagnetic ie 
focuses on examinin ‘wh = are, & Papa, 2007 ). Fractal analysis primary 
and scale dependence a Hi er time series exhibit long-range autocorrelations 
Correlations, an oheervatio a time series is said to exhibit long-range ana 

to subsequent Sheena ti made at a specific point in time will be relate 
ns in the future (Beran, 1994). Similarly, scale 
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dependence means that time se 


nes measurements (e.g., variability) differ 
as a function of the temporal 


scale at which they were measured (Man- 
delbrot & Van Ness, 1968). Fractal analyses have been used to model nu- 


merous cognitive phenomena, such as speech 
engagement (Likens, Fine, Amazeen, & Amazeen, 2015), reaction times 
(Van Orden, Holden, & Turvey, 2003), and eye movements (Aks, Zelinsky, 
& Sprott, 2002). In our own research, we have found that fractal patterns 
observed in the inter-keystroke intervals (i.e., the time between keystrokes 
during essay writing) predict essay quality (Likens et al., 2017). In addition, 
we have found that fractal patterns that characterize student choices pre- 
dict performance in an intelligent tutoring system. Importantly, research- 
ers generally suggest that variability in fractal properties of a time series 
reflect the flexibility and adaptability necessary to complete cognitive tasks 
that require the coordination of information across multiple time scales 
(e.g., Van Orden et al., 2003; Stephen, Anastas, & Dixon, 2012). 
Importantly, analytic techniques have been developed that highlight 
the important role of time in complex systems. These techniques provide 
researchers the means to both visualize and quantify important tempo- 
ral variability in data across multiple time scales. For instance, Figure 9.1 


patterns (Kello et al., 2010), 


2nd Bin 


Figure 9.1. Exam 
ume (Bin l= quic: 
ing with pauses). 


ple random walk representing one student's keystroke patterns over 
k bursts; Bin 2 = medium bursts; Bin 3 = slow typing; Bin 4 = slow typ- 
The bins represent four quadrants of interkeystroke interval speeds 
and the rainbow gradient represents the time that the keystrokes were produced. The 
dark blue represents the beginning of the essay and the red color represents the end 
Of the essay typing process. This student typed relatively consistently throughout the 
€ssay as evidenced by their walk hovering near the second bin consistently. 
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m walk visualization of an individual’s keystrokes while pen_ 
Random walk analyses are mathematical tools that B. 
used to visualize patterns in data as we coger OSs time (Benhamoy 

. Ravet, 1989; Lobry: 1996; Nelson & Plosser, ). In this example, we _ 
& Bovet, i d the intervals between keystrokes into four bins—] hej e § 
first ee typing and 4 being long pauses. Each of these catego z 
ae assigned to a vector along a basic scatter plot. Therefore, if an 
individual began typing Very quickly, the random walk would first move 
towards the first bin, whereas if they then proceeded to slow down, they 
would move down toward the 4th bin. Once each keystroke interval had. 
been assigned to a vector, we calculated a random walk for each student ‘ 
that began at the origin of the scatter plot (0, 0). For each subsequent _ 
interval between keystrokes, the walk “steps” in the direction is consistent _ 
with the assigned vector. The resulting walk represents each individual's — 
keystroke patterns over time. We additionally colored the keystrokes points 
along a rainbow gradient to represent the time that the keystrokes were 
produced. Figure 9.1 represents a student who stayed relatively consistent — 
in typing patterns over time, making small changes between bursts and 


depicts a rando 
erating an essay: 


2nd Bin 


| 


ice M=i1a) 


*= slow typing vhost Bin 2 = medium bursts; Bin 3 = slow typing: Bin 
interval speeds and the €s). The bins represent four quadrants of interkeystroke 
Were produced. The d rainbow gradient represents the time that the keystrokes 
Color represents th ark blue represents the beginning of the essay and the red 

© énd of the essay typing process. This student went throug! ~ 


erent phases ; 
of 
men quick burst typ} 
Uacross the second and bide and slow pauses as evidenced by the more 
ins. Rr 
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pauses. On the other hand, Figure 9.2 represents a student who engaged 
in very different typing behaviors—moving over time between periods of 
pauses and slow typing to periods of time where they typed in quick bursts. 
These random walk images provide an example of a way in which dynamic 
methods of visualization can be used to reveal important structure in lan- 


guage production and comprehension over time. 


Dynamical Systems Theory and Text Comprehension 


We assume that comprehension processes fluctuate over time, and a 
comprehender may fluctuate between states of understanding and misun- 
derstanding. Our hypothesis is that the transition from a state of misunder- 
standing to a state of comprehension will be preceded by critical fluctua- 
tions, and these fluctuations will be revealed through the dynamic structure 
of linguistic features of a student’s constructed response. For example, the 
use of more familiar (frequent) words has been associated with lower levels 
of language proficiency and comprehension. Prior research has relied on 
averaging word frequency across all of a student’s responses. However, fluc- 
tuations in word use may reveal when a student shifts between states of suc- 
cessful and less successful comprehension. Increases and decreases in word 
frequency may characterize shifts toward resolution of misconceptions by 
signifying that a phase transition is taking place. 

Our aim here is to promote research on the topic of text comprehension 
(e.g., evaluation of sources) that leverages concepts borrowed from DST 
such as dynamical similitude. The motivating idea is that discourse processes 
can exhibit the hallmark properties of a dynamical system (e.g., complex 
patterns; Gilmore, 1981; Kelso, 1995). In particular, our objective is to le- 
verage the assumption that we can observe and compare dynamic patterns 
across multiple time series in order to understand how complex systems 
unfold over time. In the context of fake news and misinformation, our ul 
timate objective is to explore the notion that dynamics of misinformation 
comprehension emerge in the form of complex, dynamic patterns across 
documents, constructed responses to these documents, and even lower lev- 
el data produced during comprehension tasks, such as keystroke, reading 
ume, and eye movement patterns. For instance, we assume that linguistic 
features, when extracted from complex texts or students’ constructed re- 
sponses, may provide suitable observables, which in turn can serve as impor- 
‘ant parameters in dynamical analyses. If so, they will have the potential 
to reveal themselves as a powerful lens through which to observe compre- 
hension processes, particularly as they relate to the complex processes 1n- 
volved in misinformation comprehension. As mentioned earlier, one window 
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hension is to elicit constructed responses to texts such 
: as 


to online compre 
ink-aloud. - | 3 | 
pe approach that has been used in the past is to examine the soll 
n ; 
sof each think-aloud separately and then use averages across ui 


pean feat ik quality of comprehension. For example, Magliano aid 
mance 8 ‘ eviously developed the RSAT, in which stud, 
colleagues (201 1) previ y A 7 : Thi 4! ents pro- 
vince thinkaloud responses while reading. This tool uses algorithms based 
on the presence of particular key words to analyze responses for evidence 
of comprehension processes, including paraphrasing, bridging inferences, 
and elaborative inferences. The algorithms correlate with experts’ codin 
of the protocols as well as independent measures of comprehension abil. 
ties. While RSAT algorithms provide indicators for the presence of para- 
phrasing, bridging, and elaboration during comprehension, they do not 
provide information about how they occur in combination to support com- 
prehension, nor how those processes change over time. 

Asa proof of concept, we have combined linguistic and dynamic analyses 
to understand comprehension behaviors through analyses of the temporal 
organization of constructed responses. In a recent study (Allen et al., 2017), 
students generated constructed responses (i.e., self-explanations or think- 
alouds) while reading a science text and then answered comprehension ques- 
tions. A dynamic methodology—Recurrence quantification analysis (RQA; 
Webber & Zbilut, 2005) was used to visualize and quantify the extent to which 
recurrent patterns in students’ natural language text responses relate to their 
reading comprehension processes. Recurrence quantification analysis is a 
nonlinear data analysis technique that provides information about patterns 
of repeated behavior (i.e., the number and duration of recurrences) in a 
continuous or categorical time series. Like many techniques used in the DST 
framework, this methodology has been used in a variety of domains, both 
within and outside the realm of human behavior. For example, researchers 
have utilized RQA to examine patterns of heart rate variability, postural fluc- 
tuations, and eye movements (Anderson et al., 2013). 

; Beyond these physiological measures, RQA has the potential to provide 
important information about recurrence in the content of students’ lan 
a rie and Spivey (2005), for example, have revealed that RQA can 
i rh oe data sets, such as the words in a particular Me 
applied ‘ “ exibility of the RQA technique (i.e., the fact that 1t can 
slleat fers ‘ continuous and categorical data sets) may be parte " 
sees reas a - a of natural language. In particular, recurrence aces 
relying only on 0 “7 levels of the text (e.g., word, semantic), rather t 
ithe ce ne evel of analysis. lot 
which isa eat ray of RQA is the development of a recurrence P my 
sent points in a cee of a matrix wherein the individual elements TP 
series that are visited more than once (i.e., they recU"”” 
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In other words, this plot represents the times in which a dynamical system 
visits the same area ina phase space. Within this plot, each point represents 
ri particular state that is revisited by the system, If multiple points occur con- 
tinuously, they form diagonal lines, which represent times when the system 
js revisiting an entire sequence of states. 

Asa simple illustration, consider the following sentence: “The ice cream 
man brought ice cream on Friday.” To generate a recurrence plot for this 
sentence, the words in the sentence are first placed on both the X and Y 
axes of a 9-dimensional plot (see Figure 9.3). Each time a word appears 
both the X and Y axes, a dot is placed in that location on the plot. Because 
this sentence is being plotted against itself, the recurrence plot is symmetri- 
cal with a diagonal line through the center—the line of identity (LOI). The 

oints of interest in these recurrence plots are the points that do not occur 
on the main diagonal. Individual points off the main diagonal represent the 
times that a word is repeated later in the sentence. When multiple points 
occur simultaneously, these points form diagonal lines (e.g., “ice cream” in 
Figure 9.3), which represent sequences of words that are repeated in time. 

Visualizing recurrent patterns is informative, but researchers also need 
to quantify the structure contained in recurrence plots. Recurrence quan- 
tification analysis offers multiple metrics that help to quantify recurrent 
patterns to allow for statistical comparisons of recurrence plots. Table 9.1 


the ice cream truck brought ice cream on friday 


riday 


the ice cream truck brought ice cream on fi 


re a sentence is plotted against itself and 
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igure 9.3 Example recurrence p!0 a word is repeated. 
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Metrics Used in Recurrence Quantificatio,, 


of the density of points represented in a recurrence plo. A 
recurrence rate is calculated by dividing the total number of points in 

a plot by the square of the length of the overall ume series. This metric. 
represents the overall amount of recurrence that is present in the : 
recurrence plot, regardless of the distributions of the points, 


‘A measure of the number of recurrent points that tend to fall on 
diagonal lines (ignoring the LOI) in the recurrence plot. Thus, this 
metric provides information about the distribution of the recurrent 
points. Diagonal lines in recurrence plots reflect time periods when the 
system is revisiting a particular sequence of states. Thus, systems with low 
determinism can exhibit short moments of repetitive states; however, they 
are considered less ordered than highly deterministic systems. Be ||, 


A measure 


Recurrence 
Rate 


Determinism 


Ags e Ss 
This metric calculates the average length of the diagonal lines inthe 
recurrence plot. Thus, when the system repeats a sequence of states, this 
metric provides information about the typical length of those sequences, 


Average Line 


Length 


This metric calculates the length of the longest diagonal line in the s 
recurrence plot. Therefore, this metric reveals whether a system revisits a 
long sequence of states at some point in time. : 


Maximum 
Line Length 


Entropy is calculated as the Shannon entropy of the distribution of the — 
line lengths in the recurrence plot. This metric quantifies the degree to _ 
which the trajectory of the system exhibits order. Thus, entropy will be 
higher if the system revisits a wider variety of state sequences over time. 
Dynamic systems that continually revisit the same, or similar, sequences 
of states, will have lower entropy. 


describes the most commonly used metrics in recurrence quantification 
analyses, 

To visualize the temporal distribution of words in students’ self-expla- 
nations, recurrence plots for each student were calculated using the pro 
sealing described in the previous sections. These recurrence plots varied 
considerably among the students and provided us a means to qualitatively 
Fi ai in the word recurrence in the self-explanations of stu- 

Figures sei low and high scores on the comprehension test. a 
using two oe; ae 9.5 illustrate two recurrence plots that were gen 
though the sec _actual self-explanations from the current study. a 
(Figure 9.9 = 354, a self-explanations had a similar total number of ie - 
dents exhibited differen cr plots demonstrate tn eee 
selfexplanations, €nt patterns of word recurrence throughou 

Fi ; 
tote of | tg Sa the recurrence plot of a student who rece a 
SION score = ; brid ne the comprehension test (text-based compre ie 
Plot, this student r 8ng comprehension score = 0). As illustrated in 

*rely produced self-explanations with similar words from 
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Figure 9.4 Recurrence plot for a student with a low text comprehension score. 


their previous explanations. Additionally, in the situations when this stu- 
dent did exhibit word recurrence, the words tended to occur in isolation, 
rather than in sequences (diagonal lines) of words. In other words, bi 
recurrence plot suggests that this student did not generate explicit conn 
tions between the information explained in different sections of the — 
In contrast, the plot depicted in Figure 9.5 comes from a es . cm 
received a perfect score of 8 on the comprehension test pacer me c = 
prehension score = 4; bridging comprehension score = 4). Unlike the p 
vious student, this student exhibited a high degree of recusrence ate 
self-explanations. Additionally, many of the recurrent points fell on aso 
nal lines, suggesting that this student was repeatedly referring 2 a een : 
of words, rather than individual words. Thus, while reading roug 
text, the student continued to explain the new text information in connec- 
tion with previously encountered text informauon. en 
Overall, these recurrence plots provide a means throug’ eh po aa 
prehension processes of skilled and less skilled readers can i$ 
ated. Despite the fact that these two students generated be — os 3 
of text during the self-explanation procedure, the temporat cis 
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igure 9.5 Recurrence plot for a student with a high text comprehension score. 


the words they used varied widel 
student who con tinuou 
ultimate 


y. In particular, these plots reveal that the 
iunlevelcee . repeated words and phrases while self-explaining 
the student who se Por aae rehension of the text. In comparison, 
onstrated low text c Y repeated information across self-explanations dem 
In addition ¢ i 
ated from tHeeeenn visualizations, the quantitative RQA indices genel- 
tion about students’ catia plots were able to provide important informa: 
of Correlation and re mprehension performance. In particular, the results 
students’ eS aman analyses indicated that 32% of the variance in 
of summative metric $10 scores were accounted for using a combination 
letters ber wor, EH a of word use (i.e., total number of words, the number Of 
Patterns of this word ii ae ratio), as well as indices related to recurrent 
tance of accounting fo © (see Table 9.1). These analyses speak to the impor 
alyses of atone woes patterns in analyses of students’ language: 
guage tend to rely on summative metrics of text 
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features; however, the results of the work by Allen et al. (2017) suggest that 
expanding these analyses to include temporality can provide critical infor- 
mation about students’ learning processes, 

Notably, the work in Allen et al. (2017) was the first of its kind to exam- 
ine constructed responses from a dynamical systems perspective; however, 
several other studies have reported correlations between the dynamical 
properties of comprehension (Wallot, 2017; Wallot, O’Brien, Haussmann, 
Kloos, & Lyby, 2014; Wallot & Van Orden, 2011). In addition, the applica- 
tion of DST has been broadly applied in other psychological settings (Kel- 
50, 1995; Thelen, Schdner, Scheier, & Smith, 2001; Thelen & Smith, 1994; 
Spivey, 2008) and educational domains (Koopmans & Stamovlasis, 2016). 
In combination with our own work (e.g., Allen et al., 2017), the success 
of the DST in various psychological domains leads to the hypothesis that 
the dynamical systems approach will reveal valuable information regarding 
comprehension processes, leading ultimately to improved measurement 
and interventions. 

Our overall proposal is that we can extend these types of studies to ex- 
amine students’ responses to the types of multiple and conflicting texts 
that are readily encountered on the Internet. For instance, students could 
be provided with multiple texts that contain both true and fake news in- 
formation and asked to provide constructed responses to the texts as they 
read. Our hypothesis is that the complex dynamics involved in evaluating, 
integrating, and processing conflicting sources of information can be re- 
vealed in the dynamics, such that they exhibit different patterns that can be 
quantitatively modeled. Through aggregate analyses of these responses, we 
may miss out on how students are switching amongst the texts and generat- 
ing connections amongst the text concepts across time. Dynamic models, 
on the other hand, provide researchers the means to examine the specific 
parameters and processes that govern this complex comprehension system. 


DISCUSSION 


The ability to construct a coherent mental model from multiple documents 
iS an essential skill for school and the workforce. This is particularly impor- 
tant in the Internet-age, as the texts presented to students often contain 
conflicting and even false information. Having a better understanding of 
the processes involved in these complex comprehension and production 
tasks will help us develop the instruction needed to combat misinformation 
and fake news in the modern age. We have argued that dynamic analyses 
of students’ comprehension and production processes (in the form of con- 
structed responses, keystroke patterns, text properties, etc.) can provide 
unique insights into the coherence-building processes underlying discourse 
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‘conceptions. Given that the strategies and features a 

n are constantly evolving, it is important that 
er understanding of the comptex Processes involved in reedh 
develop a dee] this information. Dynamical systems theory Provides 3 
der a variety of processes without having to focus on the in 
vidual variable in isolation of other potentially r ; 


elevant 
factors. Thus, our objective is to encourage researchers to combine current 
actors. 


: re ter un. 
sae ne as comprehension and discourse production unfold, a 
Importantly, visualizations and analyses similar to the ones presented 
throughout this chapter may be able to be used to improve tools for dis. 
course instruction. In particular, educational technologies have been 
developed to provide instruction and feedback on a variety of discourse 
production and comprehension tasks. However, these tools often focus 
on summative metrics of language use and fail to consider how students’ 
language and cognitive processes unfold dynamically. The methodological 
techniques from DST may serve as a useful addition to the technologies, 
by allowing researchers to provide visualizations and representations that 
consider how students are learning and processing information over time. 
As a final note, in this chapter, we only focused on a small subset of the 
techniques that can be leveraged from the dynamical systems “toolbox.” Ad- 
ditionally, in our analyses, we did not account for the wide variety of lin- 
guistic properties that can be calculated about a given text. We chose this 
approach because our objective is to provide a broad proof of concept about 
the power of the dynamic techniques when only surface-level features (such 
as words) are considered. However, these techniques are highly flexible and 
can be used to analyze any number of features of language. For instance, 
ap recurrence quantification analyses, such as the one presen 
oath Be alae be used to analyze recurrent patterns in the ee 
feiltine io ee documents or students constructed rae 
tists sae - Further, recurrence quantification analyses can be "de 
eye movements aan See ae ford frequency: Ker eee of 
different Fe iy y, add models can be combined using a num dale 
features interact nine a 2 account for the way inlay Here 
to conduct Studie: sie aver ame simultaneously. We urge wt icc. 
mensional Sthpere ae on prior research to account for the pie 
_ Overall, the aim se a language that students process and gener dig 
tives and methodolovic re chapter is to argue that the theoretical PO 
both theory and a 9 a! techniques afforded by DST can be used toi ne 
lar, DST emphasives Uce related to misinformation processing. In par us 
@ focus on complex systems that are not reducible t© 
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sum of their parts and which unfold in complex patterns over time. 
s theoretical perspective may provide key insights into the ways in which 
vidual process fake news, as this information is highly complex and 
inf ins multiple interacting variables that combine to influence compre- 
San and misconception resolution. Additionally, the field of DST has 
jeeiaped a number of techniques that can be used to guide both qualita- 
tive and quantitative assessments of students comprehension processes. In 
articular, these techniques can be combined with current approaches to 
tomated language analyses to model how individuals integrate and com- 
au hend the wide variety of information presented to them on the Internet. 
ae our ultimate proposal is that these dynamic visualizations and analy- 
ses can be used as a step towards more adaptive educational technologies 
for literacy, as well as for any system that automatically models language. 
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ABSTRACT 


Internet users encounter information from a wide array of sources with vary- 
ing intents and standards of publication. The Internet, which is mostly un- 
Moderated, has largely replaced sources curated by experts, such as books, 
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